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ABSTRACT 



This paper assesses the impact of business practices on 
higher education institutional research activities. Institutions who have 
been implementing total quality management practices are gradually adopting a 
new method, process redesign, which focuses more on how to achieve 
improvements using less costly research methods and procedures. This study 
outlines how one institution worked toward a research compromise. Early on, 
the institutional research office, in conjunction with quality improvement 
teams, identified a number of problem areas. It was found that: (1) quality 

improvement teams sometimes lacked research training and understanding of the 
quality management process; (2) the defined research problem was sometimes 
too large to be managed in the time assigned; (3) survey instruments were 
sometimes too inclusive; (4) the personal dynamics within and between quality 
improvement and institutional research team members sometimes created 
barriers; and (5) often, too much data were gathered. These problems were 
addressed systematically as the institution shifted focus from quality 
management to process design. (CH) 
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Avoiding a Collision Course: How to maintain research standards in the "quick and 

dirty work" of TQM and Reengineering 

Introduction 

In many ways, this paper is testimony to how much institutions can shift focus 
administratively over the period of one academic year. When we submitted our proposal to 
the Association of Institutional Research almost nine months ago, top levels of administration 
at our institution spoke mainly of the importance of total quality management (TQM) 
practices and the adoption of quality improvement techniques. The optimum word on 
campus was "customer" and the process of standardizing methods to add value for customers 
drastically shaped our improvement philosophy. 

Recently, however, a monthly column in an internal newsletter written by a top 
administrator was entitled "Is the customer dead?" Although our administrator certainly did 
not advocate a general neglect of institutional "customers," he did discuss the importance of 
external vs. internal customers in setting agendas, with external customers in our business 
understood to be the population of students. He writes: "After a decade of focusing on the 
customer, management gurus have moved on. While not entirely forgetting the customer, 
they have substituted a plenitude of other management strategies and philosophies." 

Shortly after, our institutional TQM focus moved towards a new method, entitled 
"Process Redesign." Process Redesign's initiative also focused on improvements, but 
improvements which provide ways to do business in a quality conscious but less costly 
manner, where areas of redundancy are avoided. The optimum word for the new approach 
has become "business processes" and the review of current institutional practices. 
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This reengineering approach is different from TQM in several ways. According to 
Hammer and Stanton (1995), the difference lies in the scope of the processes: 

The quality movement is essentially incremental in its world-view; it employs 
a set of problem-solving techniques whose purpose is to isolate narrowly 
confined difficulties within existing processes ... By contast, reengineering 
takes a macro-perspective. It doesn't seek to solve problems in an existing 
process but rather to discard the process entirely and replace it with something 
new (pp. 172-173). 

Although TQM and reengineering emphasize different ways of achieving 
improvements, both can greatly affect the lives of Institutional Researchers. Because the 
processes of TQM and reengineering necessitate data gathering — a point where a quality 
improvement or reengineering team must examine prior practices and assess current status — 
the expertise of an office of institutional research staff is often utilized. 

The needs of quality improvement and re-engineering teams, and the practices of an 
institutional research office may find themselves at odds, however. Data gathering for 
research projects at most IR offices necessitates systematic planning and careful analysis, yet 
the time demands of improvement teams often do not allow for weeks of research. This 
paper will examine the impact of improvement techniques like TQM and re-engineering on 
the resources of an institutional research staff and the approach that has been adopted at a 
Research I University to reach a compromise between the data needs of these teams and 
good research practice. 

Quality Improvement Methods for Higher Education 

In the past decade quality improvement techniques and other business practices have 
been widely documented in the field of higher education. Institutions use the techniques to 
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improve specific areas, such as classroom teaching practices and administrative processes, or 
the quality improvement approaches may be wide-reaching and incorporated as part of an 
overall assessment process (Sherr and Teeter, 1991; Palmer, 1996; Chabotar & Knutel; 

1997). 

Although many TQM or reengineering resource guides provide useful information on 
how to diagram process problems, translate findings into practice, facilitate 
TQM/reengineering planning groups, or sell the concept of business practices to the 
institution at large, few guides provide details or "maps" on how to gather data to inform the 
improvement process (Teeter & Lozier, 1993; Ruben, 1995). The reason for the absence of 
specific material is understandable: each problem (and thus the data needed to address the 
problem) is unique and therefore requires a distinct approach to the research design and 
subsequent data collection. 

Without some kind of prescription for data collection, however, improvement team's 
efforts may be hindered in their work by this stage of the process. Many teams lack adequate 
tools to gather the information they need to answer a research question. Simply enlisting the 
assistance of an Institutional Research Office can be problematic, though. Most trained 
researchers have well-defined approaches to the process of gathering and analyzing data. In 
the carefully dictated time limits for an improvement team to produce a resolution, data 
gathering must be quick and efficient, with timelines often not conforming to the normal 
standards used in social science research practice. The following outlines how one 
institution, through trial and error, worked towards a research compromise for some of the 
business practices adopted on campus. 
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The Adoption of TQM and the Effect on an IR Office 

As Total Quality Management techniques were the first business practices to be 
utilized on a large scale by our institution, we will illustrate primarily their effects on our 
institutional research staff and resources. In 1994, TQM principles were adopted by the 
senior level administration and techniques were modified to meet specific university goals 
with the expectation that these tools could be used by all facets of the institution.' A TQ 3 
steering committee was established to oversee the university- wide TQ 3 effort and the priority 
by which projects were addressed. 

The central coordination function for TQ 3 projects was assigned to the Office of 
Development and Training (ODT), a division within the Human Resources' organization. 

The ODT staff developed the campus-wide TQ 3 protocol for the Quality Improvement (QI) 
teams, involving seven basic process steps. As one of the stages in the standard TQ 3 process 
required a research component, ODT enlisted the assistance of the Office of Institutional 
Research for support and to serve as an in-house consultant. 

In less than two years, close to twenty QI teams were established to examine a variety 
of issues including such projects as the scheduling of the budget process, access of 
orientation information for new employees, and a re-examination of the steps involving 
pre-health advising for undergraduate students. With the exception of a few early teams, the 
Office of Institutional Research consulted or managed the research component for all QI 
teams on campus. 

Although Institutional Research staff members had experienced on-campus training 
in the philosophy and team-based techniques of TQ 3 , the process of managing the research 
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component for QI teams posed a new challenge. Each team had unique needs and a new cast 
of characters. Overall, the impact on the IR staff was enormous; in a period of less than a 
year, the project list for the office more than doubled. Typically each team would require the 
assistance of two members of the IR staff who needed to be present at all relevant meetings 
and involved in various aspects of survey instrument design, data collection, analysis and 
presentation. 

TQ 3 Teams and IR Integration 

In addition to managing the volume of new projects, the IR office needed to develop 
strategies to effectively provide services to QI teams. During the initial period, sometimes it 
was difficult to determine services that were required and like all new practices, unexpected 
problems often arose which slowed or hampered a team's progress. 

As each newly-formed QI team examined widely different problem statements and 
outcome goals, the research needs and data requirements of the groups varied accordingly, In 
addition, the research background and training of team members were disparate, ranging 
from those who had little or no experience with data analysis to those who had extensive 
training in social science or natural science research. The IR office also was placed in the 
precarious situation of maintaining an advice-giving position, yet not possessing actual 
membership on the QI teams. When offering research advice and support, it was important 
not to prescribe the type or form that the research component should take in the TQM 
process, but rather present alternative strategies and highlight their strengths and weaknesses. 

In the course of the first year working with QI teams, the IR office encountered a 




number of these challenges which fell into several broad categories: 
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1- Lack of research training/understanding : QI teams were comprised of individuals 
from a wide variety of backgrounds according to their relationship with the issue at hand. 
Some teams were fortunate to have members with strong social science or natural science 
research training, or business marketing training. However, there were other teams that did 
not have this advantage as their team members had little to no background in the process of 
gathering and analyzing data. 

The university's TQ 3 framework allows 3 to 4 weeks for initial data collection to 
inform the quality improvement process. Team members with little data collection 
experience needed to quickly become familiar with a few of the basic research techniques in 
order to move their QI team to the next stage. Once the data were collected, they also needed 
additional quick tutoring to assist them in the interpretation of data. For many of the QI 
teams, developing instruments to gather the data needed to answer research questions proved 
to be one of the most difficult aspects of the TQM process. Thus, the IR staff was faced with 
the challenge of uncovering the best way to provide a quick, broad overview of research 
methods and analysis techniques. 

2- Scope of Problem Not Manageable : On several occasions the IR staff encountered 
a team who had undertaken a research problem that was much too large to analyze in its 
entirety during such a short data collection period. The research required to effectively 
examine these "large" issues would greatly exceed the typical TQM timeline. Thus, the IR 
staff needed to help re-focus and narrow the problem to a more measurable research 
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3- Too Much of a Good Thing: Many QI teams determined that the most appropriate 
method to gather information was through a survey instrument. Several teams attempted to 
be all-inclusive and circulated drafts of the instruments to each team member, at each draft 
stage. With the Institutional Research staff coordinating the development of the instrument 
and attempting to be responsive to all team members, on occasion an instrument would be 
altered a dozen times within a one-week period to respond to the comments of all team 
members. Given the strict timeline for data collection, this painstaking process proved to be 
a roadblock towards maintaining the team's schedule. Thus the IR staff was faced with the 
challenge of making the survey development process inclusive, but not cumbersome. 

4- Political Pitfalls : Initially when IR staff members arrived on the scene to provide 
support for the research collection stage, they would be unaware of political undercurrents 
existing within an individual QI team. Although a particular method of data collection might 
appear at first glance to be the most efficient, several team members might oppose the 
method due to pre-existing political climates. Issues such as confidentiality and the 
idiosyncracies of inter-departmental cultures can create barriers regarding the types of 
questions to be asked and how responses should be collected. Thus, the IR staff was faced 
with how to avoid bending to a particular team's political influences which could result in a 
collection of data leaving many of the research questions still unanswered. 

5- Too much data! In addition to the management of instrument design and data 
collection for the TQ 3 teams, the IR office was often asked to analyze and present the data in 
a useful form. In an effort to be as helpful as possible, the IR office initially attempted to 
anticipate every possible question the team might have from the data. Early data reports 
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were quite voluminous, including a multitude of tables and graphs with the hope that if a TQ 3 
team had a question about the data, they would be able to find the answer in the report. 
Unfortunately, many of the teams were overwhelmed by the sheer magnitude of the data 
report and found that the volume of information actually impeded their process. Thus, the 
IR staff was faced with the challenge of presenting data in a more concise "user friendly" 
form. 

Using Quality Improvement in TQ 3 Research 

In order to accomodate and plan for the burgeoning needs of this new campus-wide 
endeavor, the IR staff worked with the ODT staff to continually improve their 
communication and interactions. Regular meeting times between the two staffs were 
established so that ongoing feedback on successful and less-than-successful practices could 
be discussed. 

One of the policies which emerged from these meetings was that the ODT staff agreed 
to include Institutional Research staff members earlier in the TQ 3 process, keeping them 
informed as soon as a TQ 3 proposal was received so that they could plan their resources 
accordingly. It also allowed IR staff to meet with the team leader and facilitator prior to the 
first team meeting. This earlier involvement helped avoid some of the inner QI team political 
difficulties the IR staff had encountered when it arrived late in the process. 

To address the problems of teams requesting too much data analysis or addressing 
research statements too large for the timeframe, the practice was established of having IR 
staff members meet with the QI team leader and facilitator prior to the research process to 
further clarify team needs. The team leader and facilitator also helped establish a QI 
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subcommittee to work on the detailed parts of the data collection and analysis, thereby 
avoiding the "too much of a good thing" problem encountered earlier. 

To address the problem of varied research-backgrounds of team members, the IR 
office began its consulting role by creating a "Data Collection Primer " 2 to be included in the 
TQ 3 manual. In this primer, basic research vocabulary were defined with examples, such as 
how to develop a research question, the difference between quantitative and qualitative 
research, and ways to analyze data in order to provide a basic level of understanding about 
the research process for all TQ 3 team members. Members of the IR staff met with the TQ 3 
teams to present the primer and to answer early research-related questions. Over time, the 
"Data Collection Primer" was modified to become more of a "workbook so that QI teams 
would approach the data process better prepared to discuss specific research problem areas. 

The Institutional Research Office also changed its usual approach to the presentation 
of data reports. The data presentations which were most useful to TQ 3 teams were quite 
different than those completed by the IR office for earlier institutional projects. Generally, 
the data presentations needed to be simpler, limited only to results relevant to the research 
problem and presented graphically. Team leaders were also given the option of requesting 

additional information derived from the data, as needed. 

The Movement Towards Process Redesign and the Effect on Institutional Research 

As the institution has shifted its focus away from TQ 3 and towards Process Redesign, 
the office of institutional research saw a precipitous drop in the formation of new TQ 3 teams 
As a result, TQ 3 efforts shifted more towards providing follow-up evaluations with 
pre-existing teams than with the gathering of data for newly formed teams. With the onset 

2 Copies of the "Data Collection Primer" are available upon request. 
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of Process Redesign, however, the "breathing period" for the IR office proved to be quite 
short. The beginning stages of Process Redesign have necessitated extensive data gathering 
and the institution contracted with an outside consultant to assess five major areas within the 
university. As part of the assessment, the consultants required extensive information, much 
of which was housed in the IR office. The data requests spanned many divisions within the 
university and periods of time. Fortunately, many of the areas of importance had been 
examined by research in our office at some point in time, and in several areas we had 
significant historical data for comparisons. 

At this stage, the institution is eagerly awaiting the consultant's report and this report 
should yield a series of options for the institution to consider. As Process Redesign teams are 
formed and changes instituted, there will be a need for evaluation and assessment, a potential 
new endeavor for the institutional research office. 



The adoption of business practices at the university has required considerable 
flexibility on the part of the institutional research office staff. Staff members have been 
asked to gather and provide data on extremely short notice, and to present these data in a 
useful, meaningful way. In an attempt to adapt quickly to the requirements for quality 
improvement, it was imperative to require standards of ourselves which allowed us to 
maintain and communicate research integrity. As we look forward to ever-increasing 
business practices in our midst, the more we can do to anticipate and prepare for the needs of 
the practices, the better. It has been suggested that in order to become significant players in 
the redesign of institutions that we "collect and make available information on the 
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institution's current situation, recent past and likely future," and "get to know the values, 
history, and relevant dimensions of the institution." (Dunn, 1995, pp. 108-109). 

Our office was fortunate in that we had the resources and systems in place to examine 
data historically and in ways directly relevant to the processes examined. However, even if 
quality improvement practices are not on the immediate horizon for an institution, given the 
current climate in higher education, they are likely to arrive soon. Anticipating ahead of 
time in an office of institutional research can allow better preparation and control over the 
implementation of business practices. 
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